An Efficient Transformer Decoder with Compressed Sub-layers

The large attention-based encoder-decoder network (Transformer) has become prevailing recently due to its effectiveness. But the high computation complexity of decoder raises inefficiency issue. By examining mathematic formulation decoder, we show that under some mild conditions, architecture could be simplified by compressing sub-layers, basic building block Transformer, and achieves a higher parallelism. We thereby propose Compressed Attention Network, whose layer consists only one sub-layer instead three. Extensive experiments on 14 WMT machine translation tasks our model is 1.42x faster with performance par strong baseline. This baseline already 2x than widely used standard without loss in performance.